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On the use of Kinetography Laban to notate 
robot action and motion 


Paolo Salaris, Naoko Abe and Jean-Paul Laumond 


Abstract —Roboticists aim to segment robot actions into a 
sequence of motion primitives to simplify the robot programming 
phase. Choreographers aim to capture the essence of human body 
movements within a sequence of symbols that can be understood 
by dancers. To that extent, roboticists and choreographers pursue 
the same quest. We have undertaken a pluridisciplinary approach, 
combining a dance notation system (the Kinetography Laban sys¬ 
tem) with a robot programming system [the Stack of Task (SoT)]. 
Motion scores are used instead of quantitative data to compare 
and enlighten differences in robot and human movements. We 
then discuss plausible origins of these differences, taking into 
account the implicit rules of the Kinetography Laban system on 
how a movement is executed by humans. This comparison, in the 
light of the Kinetography Laban system, opens some challenging 
questions related to motion segmentation and motion naturalness. 


I. Introduction: Dance Notation and Robot 
Motion 

A. Motion writing 

How do you park a car? A way to answer the question is to 
consider optimality principles. Starting from the seminal work 
by Dubins in 1957 [1], optimal control for wheeled mobile 
robots has attracted a lot of attention. In [2], the problem of car 
parking is solved by considering a sequence of shortest length 
paths, i.e., the so called Reeds-and-Shepp paths [3] (Fig. 1). 
The shortest paths are made of two basic maneuvers: an arc 
of a circle (on the right/left, executed in a forward/backward 
direction) and a straight line (executed in a forward/ backward 
direction). However, not all arbitrary concatenations of these 
two basic maneuvers generate optimal paths. Only some of 
them may be optimal. In other terms, there exists a finite 
family of sequences of arcs and straight lines that covers all 
possibilities to maneuver from a start ing configuration to any 
goal configuration in an optimal way. Such sequences can 
be seen as the words of a simple motion language with an 
alphabet that is made of two letters that are “arc-of-a-circle” 
and “straight-line segment”. The car parking motion can then 
be described as a sequence of words. With this perspective, 
motion planning is a matter of motion writing. 

This simple car parking example perfectly illustrates the 
challenge of robot motion planning and control. The question 
constitutes the essence of robotics: how to transform an action 
expressed in the physical space (i.e., park the car or pick up 
the ball on the floor) in terms of a sequence of motions that 
originate in the motor control space (i.e., turn left-forward, go 
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csc n/2 \c c\c\c c\c a c a \c 

The words to park a car: 

{C\C\C, cc\c, c\cc, cc a \c a c, c\c a c a \c, 

C\C W/2 SC, CSC n / 2 \C, C\C n/2 SC n/ 2\C, CSC} 

Fig. 1. The algorithm in [2] computes collision-free motions for a car¬ 
like robot. The solution to park the car is a sequence of Reeds-and-Shepp 
elementary paths. Each elementary path is a combination of arcs-of-a-circle 
C and straight-line segments S. The motion can then be written as a sentence 
from a vocabulary of nine words made of two letters, C and S. (Photo courtesy 
of LAAS-CNRS) 


straight, turn right-backward, or bend the legs and then move 
the right hand toward the ball). The segmentation of complex 
movements is a fundamental step to make robot programming 
easier. 

Human beings and humanoid robots share a common 
anthropomorphic shape. If the ultimate goal of roboticists 
is to provide humanoid robots with autonomy, a quest for 
dancers and choreographers is to understand the foundations 
of human movements. In spite of completely differ ent cul¬ 
tures and backgrounds, both communities pursue converging 
objectives. In this context, it is natural to assess the potential 
of dance notations for decomposing complex robot actions into 
sequences of elementary motions. Indeed, the main purpose of 
dance notation is to store choreographic works and knowledge 
of dance techniques by translating movements into abstract 
symbols such as letters, abbreviations, musical notations, stick 
figures, and so on. In Western cultures, there are almost 90 
dance notation systems, originating from their first appearance 
in the 15th century to the present. Among the most popular 
are the Kinetography Laban system, the Benesh movement 
notation system, and the Eshkol-Wachman movement notation 
system [4]. 

B. The Kinetography Laban system 

We opted to focus on the Kinetography Laban system. 
This notation system scores all anthropomorphic motions in¬ 
dependently of any behavior or any action and, hence, can be 






used with humanoid robots. The Kinetography Laban system 
provides a way to segment and analyze complex movements 
of humanoid robots. In doing this, we are able to lay the 
foundation for a more ambitious goal of simplifying robot 
programming by means of motion segmentation. In particular, 
we can see how the respective notions of robot action and robot 
motion (e.g., [5]) can be expressed within the same notation 
system. Our study is supported by two main experiences. The 
first is related to the execution of an action, such as picking 
up a ball. We show how the Kinetography Laban system 
may score the task at different levels of detail according to 
the objective to be reached, e.g., action execution or gesture 
imitation. It can range from a simple notation of a complex 
action to a detailed description of a simple motion. This makes 
the Kinetography Laban system a useful tool that allows us to 
capture the motion differences in executing the same action 
and, there fore, gives a measure of naturalness of the whole 
action. In our opinion, a single quantifying criterion, which 
should be chosen among the several available in the literature, 
would not reach the same objective. 

However, we also demonstrate that the Kinetography Laban 
system might be useless in spotting the differences between 
two movements without the context. This is because the 
Kinetography Laban system operates in the physical space 
while the main differences between two movements might be 
in the motor-control space. The second experience accounts 
for a dance imitation that is also reported in [6]. The dance 
score is translated in terms of a robot program, i.e., the so- 
called SoT [5]. Even if the dance movements are simple and 
not challenging for a humanoid robot, significant differences 
appear between the original human movements and the robot 
movements. Such differences are understood by comparing the 
Kinetography Laban scores that describe the human motions 
and the humanoid robot motions. The differences between 
the scores provide a better understanding of what makes a 
movement natural. 

We first review research related to the segmentation of 
complex movements, dancing robots, and computational scor¬ 
ing. We then introduce the basics of the Kinetography Laban 
system. This is followed by the first experience of picking up 
a ball. The motion performed by the HRP2 robot to pick up 
the ball is notated with the Kinetography Laban system. The 
objective is to point out the flexibility and the limitations of this 
notation system in expressing anthropomorphic movements 
with different levels of details. We then share the experience 
reported in [6] of translating the Kinetrography Laban score 
of a particular dance, known as the Tutting Dance, into a 
hierarchical sequence of tasks to be executed by the humanoid 
robot Romeo. The Kinetography Laban system is used to 
compare Romeo’s movements with the dancer’s. It appears 
that Romeos movements differ from the human ones. We will 
see how these differences might refer to recent neuroscience 
and biological studies. 

C. Segmenting complex movements 

Several reported experiments promote the idea that motor 
actions and movements in both vertebrates and invertebrates 
are composed of elementary building blocks [7]; i.e., complex 
movement is segmented into simple movements, and the com¬ 
bination of these elementary steps results in a complex action 


or movement, just the same as how the letters of the alphabet 
make up more complex ideas in the form of words. The 
motion segmentation of complex human movements is widely 
studied, and there is much research to be found in the literature. 
The main objective is to determine this alphabet of human 
movements. For instance, in [8], the authors automatically 
constructed a directed graph called a motion graph that encap¬ 
sulated connections among the data base from human motion 
capture data. Motion was generated simply by building walks 
on the graph. In [9], the role of a parameter that characterizes 
the two thirds power law was investigated. This parameter was 
nearly constant during extended parts of the movement and 
only shifted abruptly at certain points of the trajectory. This 
was interpreted as an indicator for segmented control. In [10], 
the authors showed that imagined trajectories follow the two- 
thirds power law. These findings support the conclusion that 
the coupling between velocity and curvature originates in 
centrally represented motion planning. However, for particular 
cyclic or repetitive actions, such as elliptical and figure-eight 
patterns of different sizes and orientations performed by using 
the whole arm, there is no evident segmentation in the motor- 
control space but rather continuous oscillatory patterns [11]. 

In the field of learning by demonstration, a general ap¬ 
proach for learning robotic motor skills from human demon¬ 
stration was introduced [12]. By using a nonlinear differential 
equation to be learned to represent an observed movement, 
the researchers built a library of movements by labeling each 
recorded movement according to task and con text (e.g., grasp¬ 
ing, placing, and realizing). In [13], a hierarchical framework 
capable of learning complex sequential tasks from human 
demonstrations was proposed. Through a task-segmentation 
and action-primitive discovery algorithm, both the high-level 
task decomposition and low-level motion parameterizations 
were achieved for each action. Finally, in [14], the authors 
proposed the use of nonnegative matrix factorization to address 
the problem of segmenting combinations of initially unknown 
human motion primitives associated with ambiguous sets of 
linguistic labels during training. This technique allowed the 
system to find the combinatorial structure of parallel combi¬ 
nations of unknown primitives. 

II. Dance notations 

Dance notation is to dance what musical notation is to 
music and what the written word is to drama. It is basically a 
symbolic description of human movements and forms by using 
graphic symbols and figures, numerical systems, path map 
ping, as well as letters and words. A recorded dance notation 
that describes a dance through symbols is known as a dance 
score. The most frequently used dance notation systems are the 
Kinetography Laban system, originally created and published 
by Rudolf Laban in the late 1920s, the Benesh movement 
notation system, invented by Joan and Rudolf Benesh in 
the late 1940s, and the Eshkol-Wachman movement notation 
system, created in Israel by dance theorist Noa Eshkol and 
Avraham Wachman in the late 1950s. All of them allow the 
notation of every kind of human movement [15], although 
they differ in the way they represent the human body and its 
movements. 

The Benesh movement notation system is very similar to 
the modern staff music notation. It is recorded on a five-line 



stave from left to right with vertical bar lines to mark the 
transition of time. For this reason, Benesh notation is often 
synchronized with a musical staff. It draws the position of a 
dancer as seen from behind, from the top of the head down to 
the feet. From top to bottom, the five lines of the stave coincide 
with the head, shoulders, waist, knees, and feet. The system 
uses abstract symbols based on figurative representations of 
the human body. Additional symbols are used to notate the 
dimension and quality of movements. 

Eshkol-Wachman movement notation scores are written on 
grids, where each horizontal row represents the position and 
movement of a single limb, and each vertical column rep 
resents a unit of time. Eshkol-Wachman movement notation 
represents the body as a stick figure. The body is divided at the 
joint level, and between two consecutive joints, a line segment 
is defined. A spherical coordinate system is used to relate those 
segments in three-dimensional space. Positions of the free 
end of the segment can be defined by two coordinate values 
on the surface of that sphere. Segment positions are written 
somewhat like fractions, with the vertical number written 
over the horizontal number, and the horizontal component 
is read first. These two numbers are enclosed in brackets to 
indicate whether the position is being described relative to an 
adjacent limb or to external reference points, such as a stage. 
Movements are shown as transitions between initial and end 
coordinates. 

The Kinetography Laban, or Labanotation, is a system of 
recording all forms of movement through graphic symbols. It is 
used not only by dancers to write down choreography but also 
in every field in which there is the need to record movements 
of an anthropomorphic body [16]. The Labanotation uses four 
factors to describe a movement: the parts of the body, the space 
(by using direction and level symbols), the duration, and the 
beginning and the end of the movements. 

In the Kinetography Laban system, the occurrence of 
movement is called vertical stroke or action stroke. The reading 
direction starts from the bottom, and a double line denotes the 
beginning of the movement. Any symbol before this double 
line refers to a starting configuration Lig. 2(a). An action can 
occur on the left side or on the right side of the body. To 
separate an action stroke that refers to one side of the body or 
the other, a vertical line, called the center line, is drawn and 
connected to the double starting line. 

The Kinetography Laban system uses a vertical three- 
line staff that represents the body Lig. 2(a). The center line 
represents the center line of the body, dividing right and left. 
A staff concerns the human body and its movement. Vertical 
columns on each side of the central line are used for the 
main parts of the body. As a consequence, by placing the 
movement indication in one of the vertical columns of the 
staff, a movement for a particular part of the body is defined. 
Lig. 2(b) shows which part of the body each column refers to in 
a standard staff. In Lig. 2(b), the central columns immediately 
next to the center line represent the support column. Symbols 
placed in these columns indicate progressions of the whole 
body through the transfer of weight. The second columns, just 
next to the support columns, are used to notate leg gestures, 
i.e., a leg movement that does not carry weight. These columns 
can be used also for specific parts of the legs (the thigh, lower 
leg, and foot). In the third columns, outside the three-line staff, 


symbols describing the gestures for the upper body, the torso 
and all its parts, are placed. The fourth columns, immediately 
beyond the torso columns, are for arm gestures. As for legs, it 
is possible to add columns to indicate movements of the parts 
of the arm if needed. Linally, the last column on the right, 
slightly apart from the other columns, is the head column. 

The main building blocks of the Labanotation are the 
direction symbols (Lig. 2(c)). These symbols define the spatial 
directions in which the part of the body should move to reach 
a given position. This representation of gestures suggests that, 
in the Labanotation, the final destination is more important 
than the followed path. It is important to note that the Kine¬ 
tography Laban is a movement notation system, and symbols 
represent changes in the current body configuration produced 
by a movement. As a consequence, there are no symbols to 
represent an absence of movement. 

The directions in space are specified with respect to a 
central point, which is called the place and is represented by 
a rectangle (see Lig. 2(c)). By slightly changing this symbol, 
nine main directions can also be specified with respect to the 
central point. Moreover, by using three different degrees of 
shading, three different levels (low, middle, and high) can also 
be specified (see Lig. 2(c)). The combination of the nine main 
direction symbols with the three shading levels gives rise to 
27 principal directions. Notice that the direction symbols state 
only information about the direction and level to be reached. 
Once these symbols are placed in a column of the vertical staff, 
it is possible to know which part of the body the symbols refer 
to and hence the direction and level to be reached with respect 
to the point of attachment of that body part. Lor example, 
the whole arm is attached to the body by the shoulder. The 
shoulder is the point from which all directions and levels 
radiate. The whole arm can move with respect to the shoulder 
in order to reach with the extremity, i.e., the hand, the direction 
and level stated by the symbol. The hand is considered, in this 
case, the free endpoint of the arm. 

The length of the direction symbol indicates the duration of 
movements (Lig. 2(d)). The longer the symbol is, the slow er 
the movement will be. The beginning of the symbol indicates 
the beginning of the movement, and the end of the symbol 
indicates the end of the movement. To describe other details 
about the movement executions, particular signs can be used 
for other parts of the body such as fingers, the palm, or the 
back of the hand, and for parts of the body that have to touch 
other body parts or objects in the environment. An exhaustive 
description of all these symbols and signs is available in [16]. 

We chose to use the Kinetography Laban system in our 
work for three reasons: its geometric representation of the 
space around the human body, its more intuitive symbols com¬ 
pared to other dance notation systems to describe movements, 
and its simplicity in writing/reading simple scores. These 
attributes, for the very simple movements we have described, 
make the process of translating and automatizing a score in a 
robot programming for humanoid robots easier, even for a non 
expert notator. 
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Fig. 2. The main elements to notate a movement in the Kinetography Laban system are the three-line staff with columns for gestures, (a) The three-line staff 
represents the body; the direction symbols and levels, and the duration of a movement specified by the length of the symbol, (b) The standard staff show column 
gestures of the body parts and support of the weight; the direction symbols and levels, and the duration of a movement specified by the length of the symbol, 
(c) Direction symbols and levels specify the spatial directions and level in which the part of the body moves to reach a given position, (d) The symbol length 
indicates duration. 
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Fig. 3. To pick up the ball between its feet, the robot has to step away from the ball, while humans can grasp the ball without changing the position of their 
feet. This is a consequence of the mechanical differences between the human body and the HRP-2 body, (a) Paolo Salaris can pick up the ball between his 
feet without changing his foot position, (b) HRP-2 picks up the ball between its feet after repositioning its feet; stepping away is a direct consequence of the 
action “pick up the ball”, even though there is no dedicated module in charge of stepping away, (c) Tiphaine Jahier executes a motion by reading the notation 
in Fig. 4(c), which describes the movements of HRP-2 in Fig. 3(b). For Jahier, “pick up the ball” is not an objective, but is just part of a complex motion she 
has to perform. 


III. From Simple Action Notation to Detailed 
Motion Notation 

Consider the simple action of grasping a ball on the floor 
(see Fig. 3). This relatively simple action involves complex 
motion requiring the coordination of all body segments. For 
example, the legs naturally contribute to the action, such that 
bending the knees becomes a direct consequence of the action 


“pick up the ball on the floor”. The action “pick up the ball” 
can be notated as in Fig. 4(a). The score tells us only the 
initial posture, i.e., standing, as well as the arm configuration 
at the beginning and at the end of the action, i.e., the arms 
are stretched out along the body, and the initial position of the 
ball, which is on the floor between the feet. The notation does 
not mention how to pick up the ball in detail. More over, the 
symbols on the right side of the three-line staff describe that the 






















































Fig. 4. Different Kinetography Laban scores describing the motions motivated by the action take the ball. The figures may seem obscure for readers unfamiliar 
with Kinetography Laban notation. Their purpose is mainly to show that differences appear. Moreover, the presence of a symbol modeling the ball shows that 
the notation not only deals with the movement of the human body parts, but also with the movement of the ball, (a) The notation of the action take the ball 
using the Kinetography Laban system, (b) A detailed description of the movement in Fig. 3(a) in the Kinetography Laban system, (c) A detailed description of 
the movement in Fig. 3(b) in the Kinetography Laban system. 


right hand has to reach the ball by following a direct path and 
grasp it at a given instant. This is the only information included 
in the notation. This is where the flexibility of the notation in 
describing anthropomorphic movements of our robot, HRP2, 
can be used with different levels of detail, going from directly 
translating the sentence “pick up the ball on the floor between 
feet” (Fig. 4(a)) to precisely describing the movements of all 
body segments (Fig. 4(b) and Fig. 4(c)). It can also be used 
to highlight the dif ferences between two movements guided 
by the same action score as the one in Fig. 4(a). 

A. HRP-2 takes the ball on the floor 

How should we program HRP2 to pick up the ball on 
the floor following the Kinetography Laban score reported in 
Fig. 4(a)? The critical issue is to enlarge the feasible workspace 
of the robot arms when needed, i.e., when the object to grasp 
is out of the reaching space. This is done by allowing a few 
steps. The foot placements are determined by a continuous 
deformation of a virtual robot motion. The virtual robot is 
made of the robot augmented with a virtual kinematic chain 
modeling the sequence of possible foot placements. Doing so, 
the whole-body grasping task may be solved by a classical 
inverse-kinematics algorithm [17] 

The final complete movement obtained by programming 
HRP-2 as described in [17] is shown by snapshots in Fig. 3(b). 
The main reason that the robot steps away from the ball before 
picking it up is to reach a configuration such that HRP2 can 
avoid a self-collision during the task. Moreover, from this new 
position, balance can be more easily maintained. It is important 
to note that there is no dedicated module in charge of the 
action “stepping away”, but this is a direct consequence of the 


action “picking up the ball”. In other words, as with human 
movement, the legs naturally contribute to the action. 

B. Using the Kinetography Laban to compare movements 

When we asked Salaris to pick up the ball without giving 
any constraints, he picked up the ball without changing his 
foot position (see Fig. 3(a)). He just bent his knees, and his 
right hand grasped the ball. The action of bending the knees 
is not explicitly expressed by the Kinetography Laban score 
in Fig. 4(a), just as HRP2s stepping away action was not 
explicitly expressed by the score. The simplest Kinetography 
Laban score that describes only the action pick up the ball on 
the floor does not have enough detail to describe precisely how 
a movement should be executed or to compare two different 
movements used to complete the same action; as a result, 
Salaris and HRP2 executed the same action in different ways. 
Perhaps another humanoid robot and another human subject 
would execute the same action in completely different ways. 
(We did not seek to determine the most natural movement 
for humans to pick up a ball on the floor-that is an entirely 
different endeavor. For our purposes, it is sufficient to observe 
that there are different ways to complete an action, but, without 
sufficient details in the Kinetography Laban score, it is not 
possible to predict or compare these differences.) The main 
objective of the score in Fig. 4(a) is to communicate to the 
reader the action or task to be executed; the movements behind 
the action are less important. 

A movement score can be notated with different levels of 
details to account for what the notator wants to convey to the 
performer and for the person who is reading the score. This 
is not a weakness but a strength of the notation system. For 






























































Fig. 5. On the left picture, the robot has to grasp the ball on the table in front 
of it. As the ball is far away from the robot but reachable without moving 
the feet, the robots bends forward. However, to maintain balance, the left arm 
moves behind. In the figure on the right, the robot has to grasp a ball in front 
of it and a ball behind (of course the ball behind has been intentionally placed 
at the end position of the left hand on the left picture). Is it possible to spot 
the differences between these two motions by using the Kinetography Laban?. 

instance, the score depicted in Fig. 4(b) is the detailed notation 
of Salaris’s movement. The notation describes his manner to 
take the ball with many details. It includes how he reaches 
to the floor with the hand (e.g., the rotation of the torso), 
how he grasps the ball (e.g., the choice of the right hand), the 
direction of his gaze, and the motion timing. Fig. 4(c) shows 
a notation of HRP2’s entire movement. It includes exactly the 
same level of details as the score in Fig. 4(b). By comparing 
the two Kinetography Laban scores, it is possible to appreciate 
that these two movements are different, and that the HRP2 
movement appears to be much more complex. Notice that 
HRP2 did not execute Salaris’s score because of its mechanical 
constraints/limits. 

Finally, we asked Jahier, an experienced Kinetography 
Laban notator, to read the score in Fig. 4(c) concerning the 
detailed motion of HRP2 and to perform the motion only 
according to the score (see Fig. 3(c)). she was not aware of 
the objective of the study and she did not see HRP-2 before 
executing her motion. She was not aware of the objective of the 
study, and she did not see HRP2 before executing her motion. 
For Jahier, the action “pick up the ball on the floor” is not 
an objective; rather, it is just part of a complex motion she 
has to perform. The motions executed by Salaris and Jahier 
differ, even though the underlying action is exactly the same 
in both cases, i.e., grasping the ball. Salaris’s motion reflects 
the intention to grasp the ball. His motion is not constrained by 
an objective. Jahier’s motion reflects the imitation of a motion, 
in that grasping the ball is only a side effect of the motion. 

These examples show how flexible the Kinetography Laban 
system is as a motion-notation system. According to the 
objective of the notator, the score may encode different levels 
of detail, ranging from a simple description of a complex 
action to a detailed description of a simple motion. 

From previous experience, we can deduct that the Kinetog¬ 
raphy Laban system can be used to compare humanoid robot 
actions and movements with human actions and movements. 
A single quantifying criterion, chosen from among several 
available in the literature, would not reach the same objective. 
However, it is important to recall that the Kinetography Laban 
system translates a movement in symbols by looking only at 
the physical space. This might complicate the comparison of 
movements or make it impossible a description of the context. 
In some cases, the comparison might be simpler in the motor- 
control space. Let us consider, for example, the two scenarios 
reported in Fig. 5 taken from [18]. In the first scenario, the 


humanoid robot has been programed to reach with the right 
hand the ball on the table in front of it. As a secondary effect of 
the main task, the left hand moves behind the robot to maintain 
the robots balance. In the second scenario, the robot not only 
has to grasp the ball on the table in front of it, but it also 
has to grasp a second ball behind it (the ball behind the robot 
has been intentionally placed at the end position reached by 
the left hand as a result of the movement required to maintain 
balance). By looking at the two movements of the robot, it 
is not possible to spot the differences. Only the context, i.e., 
the presence of two balls to be grasped instead of just one, 
provides some hint about the intention of the movement. 

The Kinetography Laban score describes movement as it 
appears to the notator’s eyes. Depending on the level of detail, 
however, the two movements may be notated differently. Let 
us assume that a low level of detail is used, similar to the score 
of Fig. 4(a) which describes the action of “picking up a ball on 
the floor”. We can see that, in the first scenario, the movement 
of the left arm will not be notated. Indeed, the main action to 
write down is to grasp the ball on the table; the action of the 
left arm to maintain balance is not included. 

In the second scenario, both actions are written down. If 
a high level of details is used, the movements of all parts of 
the humanoid robot will be notated, and, as they appear very 
similar, the only difference will be the presence of two balls 
in the second scenario and only one ball in the first scenario. 
By using a high level of detail, the actions of grasping the 
balls are only part of a complex movement. However, results 
in [18] show that the movements of the two scenarios can be 
distinguished in the proper task space. The presented method 
takes advantage of the knowledge of the task the robot is able 
to perform and how the motion is generated from this set of 
known controllers to reverse-engineer an observed motion. The 
method is based on the projection operation into the null space 
of a task to decouple the controllers. In other words, access 
to the motor-control space to distinguish similar look ing 
movements is exactly what the Kinetography Laban system, 
which is designed to describe human movements, is not able 
to do [18]. 

IV. On the naturalness of movements: the 

“IMPLICIT RULES” OF THE KINETOGRAPHY LABAN 

In [6], a simple Kinetography Laban score of the Tutting 
Dance sequence, i.e., a dance that mainly involves arm and 
hand movements, has been scored according to the Kinetog¬ 
raphy Laban system and translated in robot motion (Fig. 6). 
The method is based on the Stack of Tasks (SoT), a robot 
programming system introduced in [19]. The 27 principal 
direction symbols used to describe the Tutting Dance [6] are 
the starting point to translate the Kinetography Laban score in 
the SoT. In other words, depending on the current configuration 
of the humanoid robot Romeo, each principal direction symbol 
is translated as a task in the operational space. Indeed, each 
direction symbol specifies the main directions and levels with 
respect to the point of attachment of the body part to which 
the symbol refers (see Fig. 2(c)). As a consequence, with 
respect to the point of attachment, it is possible to associate 
a homogeneous transformation matrix to each symbol that 
specifies both the position and orientation of a reference frame 
at that direction and level. Based on the current position of the 






body part and the desired position specified by the principal 
direction symbols, a task function is defined as the error in 
terms of both rotation and translation between the current 
position in space of the reference frame attached to the free 
end and the desired one. The SoT software [20] is then used 
to determine suitable control signals for the motor of the 
robot such that the error becomes zero, while guaranteeing 
other tasks at the same time (e.g., maintain static equilibrium, 
maintain the static position of the parts of the body that are 
not involved in the movement, and so on). This results in a 
dynamic hierarchy of tasks. 

Once the whole movement is translated in the SoT, suitable 
control signals are sent to the motor of a simulated version 
of the humanoid robot Romeo, and the whole movement has 
been reproduced by the robot. One of the main differences 
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Fig. 6. A comparison between the Kinetography Laban score of the first part 
of the Tutting Dance executed by a dancer and Romeo. The main difference 
is in the path of the free end of the arm (here considered to be the wrist). The 
path for Romeo is along a straight line from the initial position to the final 
position (see Fig. 7). There are also several movements of the torso. Moreover, 
the arm is a little curved during the movement, and the gestures are slightly 
overlapped (the movement in the original score is a “staccato” movement), (a) 
The Kinetography Laban score for a dancer and (b) the Kinetography Laban 
score for Romeo. 

between Romeo’s movements and human movements from 
the Kinetography Laban perspective (i.e., in terms of symbols 
and signs in the Kinetography Laban score) is the arc with 
the capital letter “I” to the side of each direction symbol 
inside the arm columns (see Fig. 6 and [6] for details about 
other differences). This new symbol indicates a description 
of the path that the free end of the arm is now executing, 
i.e., a straight line. In the context of the Kinetography Laban 
system, the addition of this sign to the side of each direction 
symbol con strains the movements of the free end of the part 
of the body to which the symbol refers along a straight line. 
Without that sign, the movement adheres to the implicit rules 
of the Kinetography Laban system that, after several years of 
analysis, reflect the natural way of moving. 

The direction symbols state only information concerning 
the element of direction. Once they are placed in the appro¬ 
priate column of the vertical staff, it is possible to determine 
which part of the body has to move. Moreover, depending on 
the cur rent configuration of the body, information about the 
path that the free end of the body part to which the symbol 
refers can also be obtained, giving rise to so-called implicit 
rules. 



Fig. 7. A sequence of snapshots for the movement at step 6 in the Laban 
score of Fig. 6(a). The free end of the right arm moves along a straight line 
between the starting position and the desired one. This gives rise to other 
undesired and unnatural movements. In this case, the elbow (red point) does 
not remain at a fixed position in space. 

The information about the movement execution is achieved 
from the concept of degree-distance between direction sym¬ 
bols. Each symbol indicates a point to be reached around 
the point of attachment of the body part (e.g., the shoulder 
for the arm). Symbols that correspond to adjacent points in 
space are at a first-degree distance from one another (see 
Fig. 8). For example, if the arm moves from forward-middle 
to the adjacent right-front diagonal point, this is a first-degree 
distance. In this case, the free end of the arm, i.e., the hand, 
describes an arc-of-circle on the surface of the sphere whose 
center is the shoulder. This is called a peripheral movement 
in the Kinetography Laban system. All movements between 
first degree distance points produce this type of path. If the 



Fig. 8. Degree of distance between direction symbols. According to this 
distance, the free-end of the arm has to generate a peripheral movement, a 
transversal movement or a central movement. 

arm moves from forward-middle to side (right or left)middle, 
then the starting and ending points are at a second-degree 
distance (see Fig. 8) and the hand describes a quarter of circle 
with respect to the shoulder. In this case, we also obtain a 
peripheral movement. All movements between second-degree 
points are performed without any special flexion of the arm 
unless otherwise specified with the addition of particular signs. 

If two points are at a third-degree distance, the hand moves 
along a trajectory close to the body. This movement is not a 
peripheral path. Indeed, the arm is slightly bent, and the free 
end of the arm takes a path between periphery and center (in 
place). This is called an intermediate situation or a transversal 
movement. 

Finally, diametrically opposite points are at a fourth-degree 
distance. This is the case when the arm moves from forward- 
middle to the extreme opposite direction, i.e., backward- 
middle. The arm moves back into place, and then it extends 
outward again. These types of movements are called central 
movements in the Kinetography Laban system. 










































The control laws used to move Romeos arms do not 
generate all of these various movements. Instead, the con¬ 
trol laws implemented in Romeo simply reduce the distance 
between the current configuration of the free end of a body 
part and the desired position. The resulting path for the free 
end of a body part is a straight line with a noticeable loss in 
naturalness. To generate a straight-line path, other undesirable 
and unnatural body movements are necessary. To avoid such 
unnatural movement, an ad hoc control law that moves the free 
end of a body part along peripheral, central, or transversal 
movement might be determined. The implicit rules of the 
notation come from several years of observations, and they 
are based on the naturalness of human movements and the 
mechanical structure of the body. As is often conjectured in 
robotics, an optimality principle might be inherent in human 
movements [5]. Thus, it would be interesting to understand this 
principle, to express it in a suitable mathematic manner, and 
then to use it to determine suitable control laws for humanoid 
robots. 

One of the main limitations in translating the Kinetography 
Laban score in humanoid robots obtaining movements that 
resemble the human movements. Several biological studies 
have sought to identify the principles that explain, among all 
possible movements, the ones humans perform in every day 
tasks [21]. In [22], a robotic approach for the synthesis of 
human motion using a task-space framework is presented. In 
this framework, the authors showed that task-driven human 
motions come from the physiomechanical advantage of the hu¬ 
man musculoskeletal system under physiological constraints. 
Regarding arm movements only, the coordination of voluntary 
human arm movements was presented as an objective function 
as a measure of performance for any possible planar multi-joint 
arm movement [23]. By using dynamic optimization theory, 
the authors showed that the objective function is the square 
of the magnitude of a jerk of the hand integrated over the 
whole movement. As a result, the main objective of motor 
coordination is to reproduce the smooth est movement of the 
hand. If the arm moves on a vertical plane or in a manner 
that is not only on the horizontal plane, the force of gravity 
plays an important role. To move against gravity is a very 
different proposition compared to moving along the gravity 
vector. In [24], the authors tried to under stand how the central 
nervous system plans and controls vertical arm movements. By 
using the optimal control theory, the experimental findings can 
be explained in terms of the minimization of an optimal motor 
planning (minimum absolute work-jerk), which integrates the 
direction and the magnitude of gravity torque and minimizes 
the absolute work of forces (energy-related cost) around each 
joint. 

Another critical challenge in motor control is how the 
central nervous system deals with redundancy. One way to 
sim plify the motor control is to combine several degrees 
of freedom into synergies. In [25], by principal component 
analysis, the authors showed this behavior during fast, unre¬ 
strained, and untrained catching movements. This provides a 
reduction of the number of dimensional motor spaces into a 
few dimensional control spaces, giving rise to a simplification 
in the optimization procedure. 

Reproducing a human movement in a humanoid robot is a 
known challenge. The Kinetography Laban score system can 


be used to simplify the robot programming phase. In doing 
this, however, two issues must be taken into account. First, 
the human body and the humanoid robot body differ, and 
this influences movement. Salaris and HRP2 pick up the ball 
on the floor between the feet in different manners. The main 
reason is that the body of HRP2 does not allow it to move its 
hand between its feet in order to avoid self-collision. HRP2 
is therefore forced to step away and establish a configuration 
that avoids self-collision and maintains balance. Moreover, to 
program HRP2 to pick up the ball on the floor, we may benefit 
from the score in Fig. 4(a) but not from the other score. Indeed, 
the score in Fig. 4(c) does not take into account the mechanical 
limits of HRP-2, while the second score in Fig. 4(b) is too 
detailed and hence complex to be programmed. The level of 
details adopted in the Kinetography Laban score is a critical 
issue. Moreover, human motion notations are based on the 
kinematic structure of the human body. Adapting the notation 
to another structure is certainly possible, but it is a challenge 
by itself. Second, there is the challenge of the naturalness of 
a movement. In the Kinetography Laban system, the rules to 
move the hand in a given direction have been defined and 
described on the basis of extensive experience in observing 
human movements. These described notations, however, are 
not a priori influenced by causality principles, i.e., by the origin 
of the movement, which takes place in the muscle control 
space. It is not necessary to tell a dancer what muscles he 
or she has to activate to perform a desired movement. In 
humans, muscle activation is an unconscious process. With the 
fundamental problem of invert ing actions expressed in the 
physical space into motor controls, roboticists must address 
the causality principle. This is why, like neurophysiologists, 
roboticists try to exhibit general movement laws to explore 
plausible causality principles. Combining the minimization of 
suitable objective functions, extracted from human movement 
analysis, and the concept of synergies to reduce the variables 
to be optimized might be an interesting approach to bridge the 
gap between dance notation with their rules of naturalness and 
robot programming. 

V. Conclusions 

It is important to stress that we did not intend to propose 
a new programming system for humanoid robots. Instead, we 
sought to disseminate in the robotics community the Kinetog¬ 
raphy Laban system as a way to segment and analyze complex 
movements of humanoid robots. In particular, we have shown 
how the respective notions of robot action and robot motion 
can be expressed within one notation system and how the 
Kinetography Laban system can be used to com pare human 
and robot movements. Moreover, we have also shown how the 
SoT can be used to translate a dance score into robot motions. 
By comparing the Kinetography Laban scores that describe the 
human and humanoid robot motions, new perspectives about 
what makes a movement natural by considering the implicit 
rules of the Kinetography Laban system have been also drawn. 

We have seen that dance notation and robot program ming 
pursue two different goals in two different spaces. Dance 
notators mainly address the physical space, while roboticists 
tend to bridge both physical and motor spaces. In spite of these 
differences, the various experiences presented in this article 
may open a pluridisciplinary research perspective based on a 
mutual understanding between robotics and movement science 



as addressed by choreographers and dancers. In particular, 
the relationship between action and motion, as well as their 
symbolic and computational foundations, are complementary 
as developed by dance notation practitioners and roboticists. 
The dialogue deserves to be more deeply explored. 
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